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SUMMARY 

IFDIS  (Interactive  Fault  Diagnosis  and  Isolation  System)  is  being  developed  to  aid  in 
the  isolation  of  faults  in  the  F404  jet  engines  that  are  installed  in  the  F/A-18  fighters. 
Existing  documentation  supporting  troubleshooting  for  these  engines  is  inflexible  in  the 
level  of  sophistication  expected  of  the  user  and  it  does  not  explicitly  use  the  reasoning 
with  uncertain  information  which  is  inherent  in  human  troubleshooting.  Data,  which  is 
required  for  troubleshooting  the  F404,  is  currently  or  potentially  available  for  computer 
processing  from  a  number  of  sources.  IFDIS  will  assist  maintenance  personnel  by 
providing  on-line  access  to  relevant  information  and  will  perform  much  of  the  tedious 
interpretation  of  the  available  data.  The  expert  knowledge  embodied  in  some  of  the 
existing  maintenance  manuals  has  been  re-expressed  in  a  format  that  serves  as  the  basis 
for  the  knowledge-base  for  an  expert  system.  Expert  system  techniques  have  been 
employed  because  they  offer  benefits  of  perspicuity,  they  can  be  developed 
incrementally  and  can  cope  with  imprecise  data.  IFDIS  is  currently  based  on  EMYCIN 
but  will  be  reimplemented  using  a  commercial  expert  system  shell  in  the  near  future. 
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Chapter  1 
Introduction 


IFDIS  is  designed  to  assist  in  the  diagnosis  and  isolation  of  faults  in 
the  F404-GE-400  turbofan  engines,  two  of  which  are  installed  in  each  of 
the  F/A-18  fighter  aircraft.  Problems  with  the  engines  are  usually  brought 
to  the  attention  of  the  maintenance  crew  via  a  report  by  the  pilot  and/or 
by  the  appearance  of  a  warning  code  on  the  Maintenance  Monitor  Panel 
(MMP)  indicator  that  is  inspected  after  the  aircraft  returns  from  a  sortie.  A 
wide  variety  of  codes  can  be  displayed  on  the  MMP  indicator  relating  to  the 
aircraft’s  systems  such  as  radars,  weapons  etc.  with  only  a  subset  being 
directly  attributable  to  problems  with  the  engines  or  associated  sensors. 
When  a  problem  is  reported  by  the  pilot,  an  EE500  form  is  completed  that 
describes  the  symptoms  of  the  failure  and  the  situation  in  which  the  failure 
occurred. 

Maintenance  on  the  engines  is  performed  at  three  main  levels;  Organisa¬ 
tional  level  (O-Level)  ,  Intermediate  level  (I-level)  and  Depot  level.  O-level 
maintenance  consists  of  troubleshooting  the  engine  to  isolate  the  problem 
to  a  component  which  can  be  replaced  with  the  engine  installed  in  the 
aircraft.  This  replacement  policy  is  taken  to  the  extreme  of  replacing  the 
entire  engine  if  the  problem  cannot  be  isolated  to  a  component  which  can 
be  replaced  with  the  engine  installed.  I-level  maintenance  is  concerned 
with  repairing  modules  which  which  have  been  identified  as  fauP /  during 
O-level  procedures  and  with  troubleshooting  of  entire  engines  if  it  has  not 
been  possible  to  identify  the  problem  at  O-level.  Depot  level  maintenance 
is  focussed  on  the  repair  of  components  which  have  been  found  to  be  faulty 
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at  O-Ievel  and  I-level.  IFDIS  is  currently  concerned  with  application  to 
O-level  maintenance  but  may  be  extended  to  cover  I-level  in  due  course. 

The  F/A-18  is  a  recent  addition  to  the  RAAF  inventory.  It  is  the  first 
RAAF  aircraft  to  use  the  F404  and  consequently  no  Australian  personnel 
have  extended  experience  with  these  engines.  Training  courses  have  been 
attended  in  the  US  and  Australia  that  provide  attendees  with  an  under 
standing  of  the  principles  of  operation  of  the  engines.  Due  to  the  lack  of 
extensive  experience  with  this  particular  engine,  personnel  have  not  had  the 
opportunity  to  develop  formulaic  approaches  to  particular  types  of  failures 
which  would  result  from  having  investigated  a  number  of  similar  failures. 

The  basic  troubleshooting  documentation,  the  Work  Package  documen¬ 
tation  (i.e.  WP’s)  is  expressed  in  the  form  of  binary  fault  tree  tables  (see 
fig.  1  for  example).  A  number  of  questions  are  posed  and  can  be  answered 
Yes  or  No.  These  answers  are  used  to  trace  through  the  binary  tree  by 
selecting  the  next  question  to  be  posed.  While  the  order  in  which  tests 
are  to  be  performed  is  explicit  in  the  WP’s,  the  reason  for  the  particular 
order  chosen  is  not.  Eventually,  a  recommendation  for  remedial  action  is 
made.  These  recommendations  usually  suggest  either  adjusting,  servicing 
or  replacing  a  component.  The  appropriate  table  in  a  WP  is  selected  on 
the  basis  of  indications  of  the  initial  symptom  such  as  the  specific  MMF 
codes  or  descriptions  of  the  engine’s  behaviour  such  as  hot  start. 

The  basic  maintenance  documentation  (i.e.  the  WP’s)  is  pitched  at 
a  level  which  is  suitable  for  the  most  inexperienced  maintenance  people. 
More  experienced  people  find  the  verbose  step-by-step  approach  tiresome. 
A  more  appropriate  form  of  assistance  which  is  adaptable  to  a  range  of  skill 
and  experience  levels  is  sought. 

The  volume  of  documentation  which  is  provided  with  modern  systems 
can  be  daunting.  One  estimate  of  the  documentation  relevant  to  the  F/A- 
18  is  500,000  pages  [5]  and  simply  locating  the  relevant  section  of  this 
documentation  can  be  a  difficult  task.  Some  way  to  provide  ready  reference 
to  this  material  is  to  be  recommended.  The  example  of  the  M.D  system  [5j 
where  a  large  body  of  the  relevant  documentation  has  been  analysed  and 
cross-referenced  so  that  it  can  be  accessed  via  a  computer  using  Expert 
Systems  techniques  (see  later  section)  is  illustrative  of  an  approach  which 
has  proven  valuable. 
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All  the  above  considerations  were  important  in  deciding  that  a  new 
method  for  the  isolation,  remedy  and  recording  of  faults  should  be  con¬ 
sidered.  The  next  step  was  to  decide  what  approach  should  be  employed. 
Work  reporting  results  of  applying  Expert  Systems  techniques  to  diagnosis 
problems  indicated  that  the  use  of  such  techniques  could  prove  fruitful. 
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Chapter  2 

Expert  Systems 


Expert  Systems  utilise  techniques  developed  by  Artificial  Intelligence 
researchers.  A  large  collection  of  knowledge  is  compiled  and  applied  to 
tasks  which  are  generally  regarded  as  requiring  expertise  when  performed 
by  a  human.  An  Expert  System  is  constructed  by  capturing  the  knowledge 
of  one  or  more  experts  and  encoding  this  knowledge  in  a  form  suitable  for 
computer  processing. 

An  expert  system  usually  comprises  the  following  components  :  infer¬ 
ence  engine  ,  knowledge-base  and  user  interface.  Other  taxonomies  are 
presented  by  other  workers  but  for  the  present  purpose,  the  foregoing  will 
suffice.  The  inference  engine  of  the  Expert  System  contains  the  procedure 
for  the  selection  and  application  of  knowledge  from  the  knowledge-base.  It 
has  been  claimed  that  it  is  possible  to  separate  the  inference  engine  from  the 
knowledge-base  so  cleanly  that  one  can  have  a  number  of  knowledge-bases 
which  can  be  manipulated  by  the  same  inference  engine  for  widely  varying 
application  areas.  Other  workers  disagree  with  this  approach  and  recom¬ 
mend  that  the  builder  of  the  expert  system  start  from  a  lower  level  and 
use  a  language  such  as  Lisp  or  Prolog  to  construct  an  application-specific 
inference  engine  with  any  specialised  reasoning  strategies  needed  for  the 
application  being  built  in.  For  a  consideration  of  the  attributes  required 
of  a  shell  for  the  implementation  of  a  more  ambitious  version  of  IFDIS, 
see  the  section  titled  ’An  appropriate  shell  for  IFDIS’.  A  number  of  frame¬ 
works  for  the  construction  of  experts  systems  are  available  (e.g.  KEE,  S.l, 
ART).  Such  packages  usually  include  a  representation  for  knowledge,  often 
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via  a  rule-like  formalism,  and  tools  for  entering,  editing  and  modifying  a 
knowledge-base.  These  pre-built  packages  are  often  referred  to  as  expert 
system  shells  because  they  lack  a  knowledge-base  for  a  specific  application 
area.  The  user  provides  the  knowledge-base. 

There  are  usually  a  number  of  people  filling  different  roles  involved  in 
the  development  of  an  Expert  system  for  application  to  a  particular  prob¬ 
lem  area.  The  people  involved  in  the  development  and  use  of  an  expert 
system  are  usually  classed  into  the  following  categories  :  Domain  Experts, 
knowledge  Engineers  and  user.  The  domain  expert  or  experts  have  acknowl¬ 
edged  expertise  in  the  problem  area  under  consideration.  The  knowledge 
engineer  has  a  good  understanding  of  the  techniques  which  are  available 
for  the  construction  of  the  expert  system.  The  user  is  the  person  who  will 
use  the  completed  system  as  an  aid  in  the  completion  of  tasks  in  a  problem 
environment. 

A  variety  of  knowledge  elicitation  strategies  have  been  developed  [14] 
but  only  two  approaches  will  be  considered  here. 

The  oldest  method  is  that  of  interviewing  the  Domain  Expert  to  discover 
what  conceptual  entities  and  procedures  are  used  to  solve  the  problem.  The 
Knowledge  Engineer  and  the  Domain  Expert  build  a  knowledge-base  and 
test  its  ability  to  reach  the  correct  conclusion  for  a  set  of  examples  from 
the  problem.  The  knowledge-base  is  built  by  identifying  entities  which  are 
reasoned  about  and  also  deriving  causal  and  heuristic  knowledge  which  is 
used  to  solve  the  problem.  This  knowledge  is  then  placed  in  the  knowledge 
base.  If  knowledge  exists  in  a  codified  form,  perhaps  in  manuals  or  text¬ 
books,  then  such  knowledge  can  be  used  to  generate  a  knowledge-base  that 
can  be  modified  and  extended  with  the  assistance  of  domain  experts. 

The  more  recently  developed  method  of  inductive  rule  inference  can  also 
be  used  to  construct  a  knowledge-base.  The  Domain  Expert  supplies  a  set 
of  examples  and  the  conclusion  which  should  be  reached  for  each  example. 
An  inductive  inference  is  then  performed  on  this  set  of  examples  and  a  rule 
is  generated  which  will  reach  the  correct  conclusion  for  each  example.  This 
rule  can  then  be  used  to  reach  conclusions  about  situations  which  arise  in 
the  field.  Inductive  inferencing  can  quickly  build  and  modify  a  knowledge 
base  once  attributes  which  are  to  appear  in  the  examples  have  been  chosen. 
If  the  Expert  System  reaches  an  incorrect  conclusion  for  a  case  then  the 
knowledge-base  can  be  extended  to  cope  with  the  case  by  supplying  the 
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case  as  an  extra  example  along  with  the  correct  conclusion.  Rulemaster  and 
Expert  Ease  are  commercially  available  shells  which  provide  this  facility. 

When  the  Domain  Expert  is  satisfied  that  the  Expert  System  is  a  rea¬ 
sonable  representation  of  the  expertise  required,  it  is  handed  over  to  Users 
for  evaluation.  Often  when  a  User  is  exercising  the  system  ,  new  problems 
not  anticipated  by  the  Domain  Expert  will  arise  and  a  new  strategy  will 
be  specified  to  cope  with  the  problem.  Users  of  the  system  will  also  typi¬ 
cally  provide  feedback  on  the  acceptability  of  various  aspects  of  the  system 
and  suggest  changes  which  will  make  use  of  the  system  more  comfortable 
and/or  efficient. 

Expert  Systems  can  exhibit  advantages  over  conventional  programs  in 
a  number  of  respects.  The  expert’s  knowledge  can  be  incorporated  into 
the  system  in  a  form  that  explicitly  displays  the  expert’s  reasoning  and, 
if  requested,  can  be  presented  to  the  user  in  a  form  which  is  likely  to  be 
understood.  Typical  results  of  the  use  of  explanation  facilities  in  Expert 
Systems  can  be  found  at  points  A  and  B  in  Appendix  A.  This  differs  from 
the  conventional  program  where  many  of  the  assumptions  and  procedures 
underlying  the  solution  to  the  problem  are  implicitly  incorporated  into  the 
algorithm  used  to  solve  the  problem  and  are  not  accessible  to  users  of  the 
program. 

Techniques  for  reasoning  with  uncertain  information  have  been  devel¬ 
oped  to  minimise  problems  that  arise  when  the  user  does  not  know  the 
answer  to  a  question  or  when  the  user  is  not  totally  confident  of  the  ac¬ 
curacy  of  the  answer  supplied.  These  techniques  can  also  accommodate 
uncertainty  in  the  knowledge  provided  by  the  domain  expert.  Such  prob¬ 
lems  are  typically  not  handled  well  by  conventional  programs. 

The  problems  associated  with  the  maintenance  of  large  software  arti¬ 
facts  are  well  known.  The  use  of  Expert  System  techniques  can  lead  to 
less  error-prone  maintenance  and  enhancement  of  large  systems.  Software 
techniques  have  been  developed  to  assist  the  Knowledge  Engineer  and  Do¬ 
main  Expert  in  updating  the  knowledge-base  by  checking  for  conflicts  and 
overlap  between  pieces  of  knowledge.  Such  facilities  are  extremely  useful  if 
not  essential  when  updating  large  knowledge-bases. 

The  speed  with  which  prototype  expert  systems  can  be  developed  can 
be  of  great  benefit  in  areas  where  it  is  difficult  if  not  impossible  to  spec¬ 
ify  the  total  solution  to  the  problem  at  the  first  attempt.  The  Domain 
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Expert  is  able  to  interact  with  a  Knowledge  Engineer  and  the  Expert  Sys¬ 
tem  to  rapidly  refine  the  knowledge-base  used  by  the  Expert  System.  The 
availability  of  a  concrete  representation  (i.e.  the  knowledge-base)  of  part 
of  a  problem-solving  strategy  has  been  found  to  assist  Domain  Experts  in 
structuring  their  thoughts  when  extending  the  knowledge-base.  It  has  also 
been  noted  that  the  task  of  expressing  part  of  their  knowledge  in  the  form 
of  a  knowledge-base  can  have  an  indirect  benefit  of  improving  the  Expert’s 
appreciation  of  the  problem  domain. 

A  currently  popular  representation  for  knowledge  is  in  the  form  of  rules 
which  express  causal  and  heuristic  knowledge  which  the  Expert  applies 
during  the  search  for  a  solution  to  the  problem.  Typically,  rules  have  the 
following  format  : 

If  condition 

Then  action 

A  collection  of  such  rules  is  assembled  along  with  some  representation  of 
the  entities  about  which  the  expert  system  will  be  required  to  reason.  For 
an  example  of  an  actual  rule  from  IFDIS,  refer  to  fig.  2.  It  is  expected 
that  experience  with  the  system  in  operation  will  result  in  modifications 
being  made  to  the  certainty  with  which  the  system  reaches  conclusions.  For 
example,  the  rule  in  fig.  2  may  be  modified  to  reach  its  conclusion  with  a 
lower  certainty,  perhaps  0.8  instead  of  1.0. 

Two  primary  inferencing  strategies  are  commonly  used  to  control  the 
reasoning  which  the  expert  system  undertakes.  These  are  forward-  and 
backward-chaining.  Forward-chaining  (also  referred  to  as  Data-driven)  in¬ 
ferencing  operates  by  collecting  values  for  a  range  of  entities  and  applying 
the  rules  in  the  knowledge-base  to  these  entities.  Backward  chaining  (or 
Goaldriven)  inferencing  operates  by  establishing  a  high-level  goal  (such  as 
find  faulty  component)  and  then  determining  which  rules  can  be  used  to 
help  in  the  determination  of  this  conclusion.  This  procedure  is  applied 
recursively  until  no  further  rules  are  applicable  and  the  user  is  asked  to 
supply  a  value  for  a  particular  entity  (such  as  does  the  variable  exhaust 
nozzle  actuator  operate  smoothly  ?). 

Expert  System  techniques  will  enable  assistance  to  be  provided  to  main¬ 
tenance  personnel  in  a  number  of  ways.  Assistance  at  an  appropriate  level 
should  be  available  to  less  experienced  personnel  enabling  them  to  work  on 
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more  routine  diagnoses  without  needing  to  refer  to  a  busy  human  expert. 
Access  to  appropriate  documentation  can  be  made  quickly  and  conveniently 
available  without  the  need  to  refer  to  indexes  etc.  in  order  to  determine 
what  information  is  relevant.  The  expert  will  have  ready  access  to  proce¬ 
dures  for  the  diagnosis  of  rarely  occuring  faults  which  they  may  not  have 
personally  encountered. 


Chapter  3 
Previous  work 


A  number  of  different  approaches  to  diagnosis  of  failures  within  complex 
equipment  are  available  .  A  number  of  previous  attempts  have  been  made 
in  the  application  of  Expert  systems  to  diagnosis  problems.  The  variety 
of  domains  in  which  the  attempts  have  been  made  is  wide:  bacterial  in¬ 
fections  [3],  selection  of  drilling  mud  [16],  problems  with  automobiles  [11], 
flight  avionic  systems  [5] ,  diesel-electric  locomotives  [15],  electronic  test 
equipment  [22],  signal-switching  network  equipment  [18]  etc.  An  excellent 
guide  to  the  literature  is  provided  by  Buchanan  [4].  Many  of  the  reports 
on  this  work  claim  success  for  the  limited  problem  domain  to  which  they 
have  been  applied.  A  report  commissioned  by  ARL  on  the  appropriateness 
of  expert  systems  techniques  for  application  to  the  domain  addressed  by 
IFDIS  commented  favourably  upon  the  proposal  [12]. 

One  of  the  earliest  applications  of  expert  systems  approaches  to  diagno¬ 
sis  of  failures  in  electromechanical  equipment  was  CATS  (Computer  Aided 
Troubleshooting  System)  from  General  Electric  [15].  This  system  was  de¬ 
veloped  to  aid  in  the  repair  of  diesel-electric  locomotives.  Information  from 
various  manuals  and  knowledge  of  various  experts  in  the  diagnosis  of  prob¬ 
lems  with  locomotives  were  combined  and  incorporated  into  the  knowledge¬ 
base  for  an  expert  system  that  runs  on  a  personal  computer  and  which  can 
access  diagrams  on  video-disk  of  various  components  and  specifications  for 
use  during  repair  of  the  locomotive.  One  of  the  benefits  of  having  such  a 
system  was  the  reduction  in  the  number  of  inappropriate,  time-consuming 
and  expensive  tests  which  were  undertaken  by  less  experienced  personnel 


who  had  access  to  this  system.  The  CATS  system  is  apparently  now  in 
routine  use  within  General  Electric  . 

Another  example  of  work  on  improving  support  for  maintenance  is  that 
of  the  M.D  (Maintenance  Diagnostician)  system  [5j.  M.D  is  a  system  which 
can  trouble-shoot  the  pitch-control  channel  of  an  F-15  flight  control.  The 
system  can  provide  guidance  on  diagnosis  and  can  access  diagrams  to  assist 
maintenance  personnel.  The  system  was  constructed  to  help  the  USAF  re¬ 
duce  problems  that  arise  as  a  consequence  of  having  increasingly  complex 
equipment  to  maintain  while  the  skill-level  of  available  maintenace  person¬ 
nel  is  declining.  The  system  is  based  on  an  IBM-PC  which  can  access  a 
video  disk  which  stores  diagrams  of  parts,  schematics  of  subsystems  etc. 
The  operator  can  request  information  on  the  location  of  a  particular  part 
if  necessary  and  appropriate  test  equipment  can  be  recommended  where 
necessary.  The  knowledge  base  for  M.D.  was  constructed  by  collating  in¬ 
formation  from  a  number  of  relevant  Technical  Orders. 
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Chapter  4 
Current  work 


Work  is  currently  being  undertaken  at  Deakin  University  under  a  Re¬ 
search  Agreement  with  ARL.  An  experimental  implementation  of  IFDIS 
has  been  constructed  using  the  EMYCIN  shell  [21]  on  a  DECSYSTEM- 
20.  EMYCIN  is  based  on  the  pioneering  MYCIN  system  which  addressed 
the  medical  problem  of  diagnosing  bacterial  infections.  EMYCIN  resulted 
from  an  attempt  to  remove  all  application-specific  knowledge  from  MYCIN 
to  obtain  a  domain  independent  knowledge-base  construction  environment 
along  with  a  generally  applicable  inference  engine.  The  rule  entry  and 
editing  tools  provided  by  EMYCIN  include  a  mechanism  for  checking  for 
consistency  between  rules  as  they  are  entered.  A  limited  natural  language 
interface  capability  is  also  incorporated  in  the  EMYCIN  shell.  It  has  an 
explanation  facility  which  can  be  used  to  investigate  the  reasoning  strategy 
being  used  during  the  consultation.  Users  can  use  this  facility  to  investi¬ 
gate  why  certain  questions  are  being  asked  etc.  EMYCIN  is  primarily  a 
backward-chaining  shell  but  forward-chaining  rules  can  be  incorporated  for 
limited  data-driven  processing. 

EMYCIN  was  initially  chosen  because  of  its  low  cost  and  its  previous 
success  in  diagnostic  problem  domains.  EMYCIN  is  most  suited  to  tasks 
which  can  be  expressed  as  problems  of  structured-selection.  Structured- 
selection  problems  are  those  in  which  the  possible  solutions  to  the  problem 
can  be  enumerated  by  the  expert.  A  series  of  questions  are  then  issued  by 
the  Expert  System  and  answered  by  the  user.  Answers  to  successive  ques¬ 
tions  are  used  to  progressively  reduce  the  size  of  the  list  of  failures  which 


(11} 


could  have  resulted  in  the  observed  problem.  One  or  more  solutions  that 
are  not  excluded  by  the  answers  given  by  the  user  are  presented  at  the  end 
of  the  consultation.  In  some  cases,  the  proposed  solutions  or  recommenda¬ 
tions  are  then  ranked  in  order  o*  merit  depending  on  the  strength  of  the 
evidence  supporting  each  conclusion. 

Because  of  a  lack  of  expertise  in  the  construction  of  Expert  Systems 
and  the  difficulties  of  obtaining  further  resources  it  was  not  feasible  to 
start  by  writing  a  shell.  The  decision  was  made  to  gain  experience  in 
the  practical  aspects  of  knowledge-base  construction  and  insights  into  the 
type  of  capability  required  for  an  Expert  System  for  a  full-scale  IFDIS  by 
experimenting  with  EMYCIN. 

The  approach  taken  so  far  to  converting  the  knowledge  encoded  in  the 
fault-tree  tables  has  been  roughly  as  outlined  below.  Further  development 
will  probably  require  intensive  interaction  with  ground  maintenance  per¬ 
sonnel  and  engineers  expert  in  various  aspects  of  the  engine’s  operation.  It 
is  expected  that  such  interaction  will  result  in  alterations  to  the  procedures 
specified  in  the  fault  trees  to  better  contend  with  the  uncertainty  inherent 
in  some  judgements  that  must  be  made  during  troubleshooting  sessions. 

To  generate  the  first  EMYCIN  knowledge-base  for  IFDIS,  work  started 
with  WP  005  00  which  is  concerned  with  start  failures.  WP  005  00  is 
divided  into  three  tables,  No  Start,  Hot  Start  and  Stall  Start.  Each  of 
the  interrogatory  questions  in  the  WP  was  represented  by  a  Parameter  in 
EMYCIN  e.g.  question  a.  of  WP  005  00  Table  1  is  represented  by  a  Pa¬ 
rameter  called  FAN-COMPRESSOR-ROTATE.  Rules  were  then  written  to 
ensure  that  the  same  sequence  of  questions  was  generated  by  IFDIS  as  is 
specified  in  the  WP.  It  became  necessary  to  introduce  higher-level  Param¬ 
eters  to  the  knowledge-base  so  that  the  number  of  premises  in  rules  could 
be  kept  manageable.  The  specification  of  these  extra  Parameters  was  made 
difficult  because  of  the  concurrent  threads  of  investigation  which  can  be 
active  at  any  one  time.  Deriving  a  name  and  description  for  the  parame¬ 
ter  was,  in  many  situations,  difficult  because  the  intent  of  certain  lines  of 
reasoning  was  not  immediately  apparent.  The  initial  EMYCIN  knowledge¬ 
base  has  been  extended  to  cover  Table  1  of  WP  007  00.  The  knowledge-base 
currently  consists  of  approximately  100  rules  using  approximately  100  pa¬ 
rameters. 

EMYCIN  has  been  reasonably  successful  for  the  section  of  the  manuals 
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so  far  analysed.  The  problems  of  encoding  procedural  knowledge  have 
caused  some  difficulty  however  (see  [l]  for  a  consideration  of  this  point). 
The  quality  of  explanation  generated  from  rules  with  a  number  of  clauses 
in  the  premise  that  have  been  ordered  for  the  sole  purpose  of  forcing  a 
particular  order  of  evaluation  are  low  (see  fig.  3).  The  reason  for  the 
ordering  of  the  premises  in  the  rules  is  not  explicit  -  e.g.  no  justification 
for  the  performance  of  the  ground  idle  throttle  test  before  the  throttle  off 
test  is  given  in  RULE0C6  (see  A1  in  Appendix  A).  EMYCIN  provides  a 
facility  whereby  text  can  be  associated  with  a  rule  as  a  justification  for 
the  rule  but  this  text  is  static  and  is  not  updated  by  the  inference  engine 
as  the  results  of  tests  become  known  -  a  full  justification  of  the  rule  for  all 
situations  in  which  it  might  be  invoked  would  be  cumbersome  and  extremely 
difficult  to  understand  for  rules  with  a  large  number  of  premises..  It  was 
found  to  be  nxessary  to  introduce  higher-level  parameters/goals  so  that  the 
consultation  could  be  given  more  structure.  For  example,  the  parameter 
THROTTLE-OR-PUMP-PROBLEM  was  introduced  to  collect  the  results 
from  five  questions  (i.e.  questions  g,  h,  i,  w,  and  y  in  WP  005  00  Table 
1).  By  the  inclusion  of  higher-level  parameters/goals,  it  became  possible  to 
specify  rules  of  a  strategic  nature  (e.g.  check  fuel  supply,  check  electrical 
system)  rather  than  rules  which  are  a  collection  of  tactics  (e.g.  check  fuel 
filter,  check  fuel  pump,  ...  ,  check  yellow  cable,  check  blue  cable  etc.).  The 
identification  of  the  strategies  employed  in  the  WP’s  has  been  difficult  and 
will  be  examined  later  in  the  paper. 

The  knowledge  representation  capabilities  of  EMYCIN  were  barely  ad¬ 
equate  for  the  task  but  more  flexibility  would  have  been  advantageous. 
The  availability  of  a  richer  representation  formalism,  such  as  a  flexible  im¬ 
plementation  of  frames,  and  a  more  flexible  inference  engine  would  have 
enabled  a  better  structuring  of  the  knowledge-base.  As  mentioned  before, 
the  availability  of  a  better  method  for  the  encoding  of  procedural  knowl¬ 
edge  would  also  have  been  advantageous.  Refer  to  the  later  section  entitled 
’An  appropriate  shell  for  IFDIS’  for  further  consideration  of  desired  shell 
attributes. 

In  an  attempt  to  impose  some  discernible  structure  on  the  consultation 
given  by  the  expert  system  it  has  been  necessary  to  introduce  higher-level 
entities  into  the  knowledge-base  which  depend  on  the  results  of  a  number 
of  the  questions  which  appear  in  the  fault-tree  provided  within  the  work 
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package  documentation.  As  a  consequence  of  the  manner  in  which  the  fault- 
trees  have  been  laid  out,  it  is  sometimes  difficult  to  determine  the  reason  for 
asking  the  questions  in  the  order  specified  by  the  fault-tree.  The  reasoning 
appears  to  jump  between  investigations  in  support  of  various  competing 
hypotheses  as  to  what  component  is  at  fault.  The  sequence  of  tests  might 
also  be  partially  ordered  by  a  consideration  of  the  cost  of  performing  the 
various  tests.  Limited  use  of  experts  has  been  made  in  an  attempt  to 
clarify  some  of  the  reasoning  involved.  A  trip  to  the  Williamtown  base 
and  discussions  with  some  of  the  maintenance  personnel  there  also  helped 
to  clarify  some  issues  and  suggested  aspects  of  the  system  which  required 
further  investigation. 

It  is  not  known  how  the  fault-tree  tables  were  compiled.  It  seems  at 
least  possible  that  they  were  computer  generated  and  optimised  in  some 
sense,  perhaps  to  minimise  the  number  of  tests  to  be  performed.  Such 
optimisation  may  have  contributed  to  the  apparent  opacity  of  some  of  the 
recommended  procedures.  Access  to  the  reasoning  which  was  behind  the 
construction  of  the  fault-trees  would  greatly  enhance  the  quality  of  the 
knowledge-base  upon  which  IFDIS  is  based. 

It  is  essential  to  have  domain  experts  available  during  the  development 
of  the  knowledge-base  to  explain  the  reasoning  underlying  the  fault  trees. 
These  experts  MUST  be  able  to  answer  questions  quickly,  and  preferably 
from  direct  experience,  since  the  delay  in  finding  the  answers  to  mundane 
questions  by  extensive  reference  to  the  documentation  is  intolerable.  The 
hypothesis  that  the  knowledge  encoded  in  the  WP’s  should  have  given 
the  project  a  good  start  was  partially  correct.  The  availability  of  a  large 
amount  of  information  in  this  relatively  easily  digested  form  was  an  advan¬ 
tage  during  the  initial  construction  of  the  knowledge-base.  The  existence 
of  such  a  large  body  of  explicitly  encoded  knowledge  indicated  that  there 
existed  a  sufficient  foundation  upon  which  a  knowledge-base  could  be  con¬ 
structed.  It  is  a  disadvantage  from  the  point  of  view  of  user  acceptance 
because,  given  the  current  state  of  development  of  the  knowledge  base,  it 
is  not  immediately  apparent  to  maintenance  personnel  that  IFDIS  would 
be  any  more  useful  than  the  original  documentation.  The  only  apparent 
advantage  enjoyed  by  the  current  version  of  IFDIS  over  some  other  docu¬ 
mentation  systems,  which  have  been  disparagingly  referred  to  as  electronic 
page-turners,  is  the  provision  of  explanations  of  the  reasoning  behind  the 
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current  line  of  inquiry.  It  is  probable  that  if  the  WP’s  had  not  existed  and 
the  same  information  had  been  extracted  from  the  available  maintenance 
personnel  and  other  domain  experts,  then  the  current  IFDIS  would  be  seen 
as  a  significant  achievement.  The  addition  of  capabilities  which  will  allow 
IFDIS  to  collect  and  analyse  data  automatically  will,  of  course,  provide 
assistance  to  maintenance  personnel  which  no  documentation  system  alone 
can  provide. 
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Chapter  5 

Prototype  development 


The  version  of  IFDIS  developed  to  date  is  experimental.  A  prototype 
version  will  require  an  expansion  of  the  scope  of  the  knowledge-base  along 
with  attention  to  human  interface  quality.  During  the  construction  of  this 
experimental  version,  a  number  of  lessons  have  been  learned  which  should 
aid  the  implementors  of  a  prototype  version  of  the  system. 

It  is  proposed  that  the  prototype  system  utilise  both  forward-  and 
backward-chaining  to  provide  flexibility  for  the  utilisation  of  both  automaticall 
recorded  and  user-  supplied  data.  As  far  as  is  practicable,  as  much  use  of 
the  automatically  recorded  data  should  be  made  before  the  maintenance 
personnel  are  asked  to  perform  any  tests.  This  consideration  might  mean 
that  some  of  the  recommended  tests  are  applied  in  an  order  which  differs 
from  that  specified  in  the  WP’s.  For  instance,  if  a  test  result  can  be  ob¬ 
tained  from  an  automatic  device  very  quickly  and  cheaply  then  perhaps 
such  a  test  should  be  performed  before  even  simple  visual  checks  by  the 
maintenance  personnel. 

ft  seems  that  the  quality  of  the  user  interface  could  be  crucial  in  de¬ 
termining  the  acceptability  of  the  system.  Since  the  interface  will  be  so 
important,  a  number  of  configurations  might  possibilty  be  evaluated.  The 
evaluation  could  proceed  by  having  a  variety  of  interfaces  working  with 
the  same  knowledge-base  as  far  as  possible.  Maintenance  people  could  be 
requested  to  use  and  comment  upon  the  various  interfaces  available.  If 
it  is  not  possible  to  evaluate  a  number  of  configurations  in  parallel,  it  is 
suggested  that  the  user  interface  consist  of  a  good-quality  graphics  screen 


combined  with  a  mouse-based  interaction  paradigm.  Only  passive  graphics 
of  diagrams  from  the  manuals  which  can  be  accessed  as  required  would  be 
provided. 

It  is  suggested  that  the  prototype  system  use  the  WP  documentation 
as  a  starting-point  for  the  knowledge-base.  The  knowledge  inherent  in 
these  manuals  would  require  restructuring,  modification  and  supplementing 
so  that  the  quality  of  explanations  could  be  enhanced.  The  prototype 
system  should  utilise  a  knowledge-base  of  the  symptom-conclusion  type 
with  no  attempt  made  to  incorporate  deeper-level  reasoning  capabilities  at 
this  stage.  The  WP’s  give  no  guidance  on  the  course  to  be  followed  if  the 
answer  to  a  question  is  unknown.  Domain  experts  will  need  to  provide 
an  indication  of  the  situations  in  which  data  might  not  be  available  and 
the  action  to  be  taken  in  such  situations.  It  may  also  become  apparent 
that  domain  experts  are  actually  making  extensive  use  of  reasoning  under 
uncertainty  rather  than  using  the  strictly  categorical  methods  specified  in 
the  WP  documentation. 

It  must  be  possible  for  users  of  the  system  to  pass  comments  and  sug¬ 
gestions  on  to  the  Knowledge  Engineer  when  new  situations  not  catered  for 
by  the  knowledge-base  arise  or  when  new  techniques  are  developed  or  errors 
are  found.  In  the  situation  where  IFDIS  systems  are  in  use  at  widely  sepa¬ 
rated  sites,  the  transfer  of  such  comments  and  suggestions  to  a  knowledge 
engineer  located  at  a  central  site  could  perhaps  be  achieved  via  messages 
forwarded  by  an  electronic  mail  system.  Users  could  submit  comments  from 
within  the  consultation  as  the  need  arises  and  such  comments  could  be  for¬ 
warded,  together  with  sufficient  information  on  the  context  within  which 
the  comment  was  made  for  the  Knowledge  Engineer  to  decide  on  any  nec¬ 
essary  changes  to  the  knowledge-base.  Many  commercially  available  shells 
provide  facilities  for  the  capture  of  user  comments  but  the  interfacing  of 
the  shell  to  an  electronic  mail  system  would  require  extra  work. 

Opportunistic  redirection  of  the  consultation  will  also  be  required.  The 
user  should  never  be  required  to  answer  questions  that  they  know  to  be 
irrelevant.  As  an  extreme  example  of  the  necessity  for  this,  consider  the  case 
where  the  maintenance  person  is  attempting  to  determine  why  the  engine  is 
not  developing  full  power.  If  the  engine  suddenly  bursts  into  flames  and  the 
fire  is  successfully  quelled  it  is  unlikely  that  a  continuation  of  the  attempt 
to  improve  the  performance  of  the  engine  will  be  attempted.  The  need  for 
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the  user  to  redirect  the  consultation  might  also  arise  if,  during  a  test  or 
inspection  recommended  by  the  Expert  System  to  check  on  the  correctness 
of  an  hypothesis  which  seems  to  explain  the  failure,  another  problem  is 
noticed  that  would  explain  the  observed  fault,  then  the  user  should  be 
able  to  volunteer  this  information  and  should  not  be  asked  questions  which 
are  now  irrelevant.  The  Expert  System  should  then  check  all  available 
information  to  confirm  that  the  cause  of  the  problem  has  been  found.  If  the 
evidence  supplied  to  support  the  user’s  conclusion  was  not  conclusive,  the 
Expert  System  should  be  able  to  suggest  further  tests  which  could  confirm 
the  diagnosis.  A  full  mixed  initiative  dialogue  is  what  would  ultimately  be 
aimed  for  with  control  of  the  dialogue  alternating  between  User  and  Expert 
System  as  appropriate. 

A  major  emphasis  of  the  prototype  IFDIS  should  be  to  demonstrate  the 
potential  for  integration  of  data  from  a  variety  of  sources.  The  sources  used 
by  the  prototype  should  be  those  which  do  not  require  realtime  recording 
by  the  IFDIS  system  itself.  Complications  arising  from  the  transfer  of  data 
between  IFDIS  and  data  sources,  such  as  CA.MM,  might  be  reduced  if  an 
interface  file  is  used  to  minimise  the  amount  of  direct  interaction  between 
systems.  The  provision  of  a  prototype  system  which  allows  for  the  coherent 
utilisation  of  data  from  a  number  of  sources  will  demonstrate  that  IFDIS 
does  offer  an  advantage  over  the  traditional  methods  and  should  contribute 
to  user  acceptance. 
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Chapter  6 

Proposed  extensions 


Considerable  systems  for  computerised  support  for  maintenance  of  the 
F/A-18  aircraft  already  exist.  The  Computer  Aided  Maintenance  Manage¬ 
ment  (CAMM)  part  tracking  system  and  Maintenance  Data  and  Service 
Life  Monitoring  System  (MD&SLMS)  supported  by  the  Unit  Computing 
Facility  (UCF)  within  each  maintenance  unit  could  provide  valuable  input 
to  the  IFDIS  system.  The  ready  availability  of  the  data  from  the  UCF 
should  help  to  minimise  the  amount  of  manual  troubleshooting  which  has 
to  be  performed  on  the  ground  (see  Appendix  B  for  an  analysis  of  the  pri¬ 
mary  sources  of  answers  to  various  questions).  Maintenance  Signal  Data 
Recording  Set  (MSDRS)  data,  which  includes  the  IECMS  data,  is  currently 
collected  on  the  UCF  and  access  to  this  data  will  be  very  important  for  de¬ 
velopment  of  the  full  capability  for  analysis  of  all  data  which  is  relevant  to 
the  maintenance  function.  The  exact  use  which  will  be  made  of  the  auxil¬ 
iary  data  has  not  yet  been  determined.  Actual  and  potential  uses  will  need 
to  be  investigated  in  collaboration  with  Australian  personnel  because  the 
American  documentation  so  far  sighted  makes  no  reference  to  such  auxiliary 
data. 

At  this  stage,  it  is  envisaged  that  IFDIS  will  have  access  to  UCF,  and 
thereby  CAMM  and  MD&SLMS,  as  well  as  CDU  and  AFTA  via  commu¬ 
nication  links. 

The  F/A-18  has  an  In-flight  Engine  Condition  Monitoring  System  (IECMS) 
which  can  be  used  to  record  the  values  for  various  engine  parameters. 
Recording  by  IECMS  is  initiated  whenever  the  value  of  one  of  the  engine 


parameters  exceeds  pre-set  limits  or  when  requested  by  the  pilot  of  the 
aircraft  who  pushes  a  button  to  indicate  that  recording  should  commence. 
The  data  provided  by  IECMS  is  useful  during  attempts  to  troubleshoot  the 
engine  on  the  ground  after  a  fault  has  been  reported.  Pre  and  post-events 
records  provided  by  IECMS  can  be  evaluated  to  provide  a  limited  his¬ 
tory  of  the  behaviour  of  the  engine  before  and  after  the  incident.  Another 
record,  the  Flight  Incident  Record,  provides  information  on  the  operation 
of  the  aircraft  and  engines  throughout  the  flight.  Relationships  between 
the  recorded  values  of  various  engine  parameters  can  often  give  clues  to  the 
environment  within  the  engine  during  the  incident  that  will  aid  in  hypoth¬ 
esising  a  chain  of  events  that  could  explain  the  observed  behaviour  of  the 
engine.  Comparisons  between  parameter  values  from  each  of  the  engines 
can  be  useful  in  deciding  if  the  problem  might  have  been  due  to  atmospheric 
anomalies,  the  result  of  the  operation  of  the  aircraft  (e.g.  manoeuvreing) 
or  whether  the  problem  is  confined  to  a  malfunction  within  one  engine. 

The  CAMM  system  will  provide  histories  for  each  of  the  components 
and  modules  which  comprise  the  engine.  Using  data  such  as  the  history 
of  a  particular  main  fuel  control  unit  for  instance,  it  will  be  possible  to 
alter  the  diagnostic  strategy  for  an  engine  using  a  particularly  troublesome 
main  fuel  control  unit  to,  perhaps,  investigate  the  correct  operation  of  this 
component  earlier  in  a  troubleshooting  session  than  would  otherwise  be  the 
case.  Such  an  alteration  of  strategy  could  be  achieved  by  having  meta-rules 
which  detect  the  presence  of  any  historically  troublesome  modules  and  in¬ 
vestigate  them  first.  For  such  alterations  of  strategy  to  be  feasible,  it  will  be 
necessary  to  have  modular  descriptions  of  investigations  to  check  the  cor¬ 
rect  operation  of  particular  modules  so  that  sensible  sequences  of  tests  can 
be  specified.  Causal  models  of  the  operation  of  the  engine  or  comprehen¬ 
sive  knowledge  provided  by  domain  experts  would  be  necessary'  to  support 
more  sophisticated  reasoning  behind  reordering  of  tests  and  rules.  Data 
from  CAMM  will  also  indicate  the  particular  version  or  model  of  each  com¬ 
ponent  in  the  engine  and  it  may  be  necessary  to  modify  the  troubleshooting 
strategy  depending  on  the  idiosyncracies  of  particular  equipment  models. 

Given  access  to  the  CAMM  system,  it  should  also  be  possible  to  main¬ 
tain  a  watch  on  the  frequency  with  which  various  components  fail  and  to 
use  such  data  to  alter  the  order  in  which  questions  are  asked  during  the 
consultation  if  a  particular  type  of  component  is  found  to  be  failure-prone. 
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The  remarks  above  concerning  reordering  of  questions  also  apply  here. 

Automated  test  equipment  could  provide  valuable  input  to  IFDIS.  The 
Control  Diagnostic  Unit  (CDU)  is  an  automated  test  unit  which  can  be 
used  to  check  that  the  Electrical  Control  Assembly  (ECA)  is  functioning 
correctly.  The  availability  of  the  CDU  and  other  automated  test  equip¬ 
ment  may  lead  to  modification  of  the  troubleshooting  procedures  in  the 
WP’s  with  time-cosuming,  manual  tests  that  were  performed  late  in  the 
troubleshooting  procedure  being  replaced  by  equivalent  tests  performed 
early  in  the  procedure  by  automated  test  equipment. 

A  number  of  the  tests  required  by  the  fault-tree  tables  need  real-time 
access  to  the  engine  while  it  is  in  operation.  Readings  such  as  Exhaust 
Gas  temperatures  and  readings  shown  by  various  cockpit  instruments  are 
needed  during  troubleshooting.  Some  work  has  already  been  done  with  the 
use  of  real-time  data  for  diagnosis  [6]  [9|.  Presumably,  substantial  real¬ 
time  software  will  be  needed  to  provide  access  to  this  data.  Such  software 
might  be  run  on  the  hardware  which  also  runs  IFDlo  or  could  be  on  separate 
hardware.  The  AFTA  system  [13]  can  access  information  which  is  displayed 
on  cockpit  instruments  -  IFDIS  may  be  able  to  communicate  with  AFTA 
although  details  of  interfacing  to  this  device  are  yet  to  be  determined. 

A  greatly  enhanced  user  interface  is  also  planned.  Other  work  ([10] 
[20])  has  emphasised  the  need  to  make  the  quality  of  the  user  interface 
high  so  that  it  will  be  accepted  by  users  and  this  is  expected  to  also  be 
important  for  the  acceptance  of  IFDIS.  One  of  the  points  raised  by  techni¬ 
cians  in  the  work  reported  in  [10]  was  that  they  felt  that  finding  schematic 
diagrams  relevant  to  the  problem  under  investigation  was  a  waste  of  time. 
A  computer  system  which  could  efficiently  and  conveniently  display  dia¬ 
grams  relevant  to  a  particular  component  or  system  of  the  engine  would 
be  necessary.  The  interface  would  probably  take  the  form  of  diagrams  of 
some  type  (probably  stored  on  video  disk),  a  mouse-driven  screen  interface 
and  possibly  a  speech-synthesis  unit  for  the  guidance  of  the  maintenance 
people  while  they  are  actually  inspecting  the  engines.  It  might  be  possi¬ 
ble  to  accept  responses  to  synthesised  questions  via  a  speech-recognition 
unit  since  the  consultation  could  be  structured  so  that  only  simple  answers 
would  be  required.  The  noisy  environment  in  which  the  troubleshooting 
work  is  carried  out  may  preclude  the  use  of  this  last  option  however. 

On-line  statistics  should  allow  for  improvement  to  the  system  as  ex- 


perience  is  gained.  A  record  could  be  kept  of  the  frequency  with  which 
each  piece  of  knowledge  is  invoked  and  the  failure  rates  of  the  components. 
Questions  pertaining  to  components  that  the  statistics  indicate  fail  rel¬ 
atively  often  can  be  introduced  early  in  the  the  consultation  so  that  the 
number  of  focusing  questions  required  for  the  most  common  types  of  failure 
is  kept  to  a  minimum.  Such  an  update  to  the  knowledge-base  will  normally 
be  performed  by  the  Knowledge  Engineer  maintaining  the  system. 

The  acquisition  of  the  knowledge-base  for  IFDIS  using  manual  tech¬ 
niques  will  be  tedious  to  say  the  least.  Automated  knowledge  acquisition 
would  be  extremely  useful  during  the  creation  and  augmentation  of  the 
knowledge-base.  The  procedures  to  be  used  in  acquiring  this  knowledge 
would  need  much  work  to  perfect  and  this  could  become  a  major  direction 
of  research  for  the  IFDIS  project.  Some  current  work  which  could  have  rel¬ 
evance  to  this  facet  of  the  project  are  rule  induction  strategies  of  [14]  and 
the  work  of  Boose  [2]  with  personal  construct  theory.  It  is  expected  that 
the  initial  knowledge-base  will  be  constructed  with  the  aid  of  engineering 
experts.  A  major  source  of  the  information  which  will  be  needed  to  im Drove 
the  quality  of  the  knowledge-base  and  the  usability  of  IFDIS  will  ,  however, 
be  the  feedback  obtained  from  the  maintenance  personnel  using  the  system 
on  a  day-to-day  basis. 

The  level  of  the  consultation  should  match  the  level  of  sophistication  of 
the  user.  A  very  experienced  maintenance  person  and  someone  just  out  of 
training  will  require  different  levels  of  explanation  and  verbosity  in  prompts 
and  explanations.  The  level  of  the  interaction  could  be  user  selectable  with 
dynamic  variation  of  the  level  during  the  consultation.  To  determine  an 
appropriate  dynamic  setting,  the  system  could  analyse  the  requests  for 
elaboration  and  clarification  by  the  user  and  try  to  automatically  provide 
the  appropriate  level  of  verbosity  for  the  remainder  of  the  consultation.  A 
particular  user’s  level  of  expertise  could  be  stored  for  recall  in  subsequent 
sessions. 

The  scope  for  incorporation  of  various  types  of  models  of  the  engine  to 
aid  maintenance  personnel  in  evaluation  of  a  particular  engine  seems  to  be 
wide.  Such  models  could  fall  into  two  broad  categories;  quantitative  and 
qualitative. 

Quantitative  models  or  simulations  could  be  used  to  provide  an  anal¬ 
ysis  tool  for  predicting  what  values  certain  parameters  should  assume  for 
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par*  icular  control  inputs.  For  example,  given  a  particular  throttle  lever  set¬ 
ting,  temperature  and  altitude,  predict  the  speed  which  the  engine  should 
reach.  Such  evaluations  could  be  used  to  identify  when  the  performance  of 
a  particular  engine  becomes  marginal  even  though  it  still  operates. 

Since  the  results  of  some  complex  models  can  be  extremely  difficult 
to  interpret,  it  may  be  necessary  to  provide  a  module  within  the  expert 
system  to  aid  the  maintenance  person  in  the  interpretation  of  the  results 
of  the  simulation. 

Qualitative  models  should  be  able  to  provide  the  user  with  a  tool  for 
evaluating  what  if  proposals  -  what  if  the  electrical  connection  between 
the  thermocouple  harness  and  the  main  fuel  control  (if  there  is  such  a 
connection)  were  to  become  discontinuous  ?  What  symptoms  would  be 
observable  ?  What  tests  would  be  appropriate  to  support  or  refute  such 
an  hypothesis  ?  The  availability  of  such  a  qualitative  model  embodying 
connectivity  of  components,  qualitative  relationships  between  parameter 
values  etc.  should  also  provide  a  means  for  dramatic  improvement  in  the 
quality  of  explanations  provide  to  the  user  of  the  system.  It  is  believed  that 
models  of  this  type  are  implicit  in  the  fault  trees  but  explicit  specifications 
of  them  have  not  yet  been  obtained. 

The  incorporation  of  models  which  embody  such  deep  knowledge  of  the 
problem  domain  is  not  yet  well  developed.  A  number  of  efforts  are  under 
way  to  determine  the  techniques  which  will  be  needed  to  support  this  type 
of  reasoning  about  physical  artifacts  [8]. 


Chapter  7 

An  appropriate  shell  for  XFDX3 


It  has  become  clear  from  the  work  already  done  that  EMYCIN  is  not  a 
suitable  shell  upon  which  to  base  a  more  ambitious  version  of  IFDIS.  The 
main  deficiencies  discovered  in  EMYCIN  for  the  IFDIS  task  include  a  lim¬ 
ited  selection  of  representation  formalisms,  restricted  inferencing  strategies, 
no  realistic  support  for  mixed  initiative  dialogue  and  also  implementation 
issues  such  as  the  need  for  a  large  mainframe  and  relatively  poor  quality 
of  the  code  regarding  maintainability.  These  latter  implementation  issues 
are,  to  seme  extent,  surmountable  with  the  purchase  of  a  commercial  im¬ 
plementation  of  an  EMYCIN-like  shell,  a  number  of  which  are  available 
(e-g  S.I). 

Given  the  current  design  expectations  for  the  role  and  scope  of  IFDIS, 
it  is  expected  that  interfacing  of  the  IFDIS  to  existing  software  will  be 
required  and  the  ability  to  interface  to  a  number  of  different  computer  lan¬ 
guages  will  also  be  useful  when  new,  special  purpose  software  is  required  for 
tasks  such  as  simulation/modeling,  communications  with  various  hardware 
devices  (e.g.  CDU)  and,  perhaps  high  speed  graphics. 

The  anticipated  delivery'  environment  will  ,  of  course,  have  a  major  im¬ 
pact  on  the  choice  of  shell.  Some  of  the  currently  available  shells  require 
special  hardware  support  although  there  is  a  recent  trend  toward  providing 
systems  which  can  be  delivered  to  the  end-user  using  more  widely  avail¬ 
able  and  cheaper  hardware  such  as  PC’s.  Some  systems  require  specialised 
hardware  to  support  the  development  environment  but  allow  for  delivery 
systems  to  run  on  PC’s,  perhaps  using  a  larger  mini  for  execution  of  the 
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computation-intensive  components  of  the  software. 

It  seems  certain  that  the  shell  selected  for  the  implementation  of  IFDIS 
will  need  to  exhibit  the  following  features 

•  •  forward  chaining,  backward  chaining  and  the  facilities  for  incorpo¬ 
ration  of  user-written  inference  strategies 

•  •  a  good  representation  for  procedural  knowledge  which  can  be  in 
a  format  which  is  easily  understood  by  the  user  (i.e  will  NOT  be 
displayed  in  raw  LISP) 

•  •  access  to  the  base-language  of  the  shell  such  as  LISP,  Prolog  etc. 

•  •  a  rich  set  of  representation  methods  -  e.g.  frames,  rules  and  logi 

•  •  support  for  reasoning  under  uncertainty  (probably) 

•  •  non-monotonic  logic  support  -  perhaps  via  multiple  worlds  (as  in 
KEE)  that  competing  hypotheses  can  be  concurrently  investigated 

•  •  graphics  capabilities  -  perhaps  only  require  ability  to  display  st 
raster  images  which  are  stored  on  disk 

•  •  interface  capabilities  -  call  routines  written  in  different  languages  - 
run  on  hardware  which  can  be  interfaced  to  testing  equipment 

•  •  good  knowledge-base  construction  and  maintenance  facilities 

•  •  adequate  performance  re.  response  time  when  manipulating  the 
large  knowledge-base  which  wiil  be  required 

•  •  support  for  a  range  of  levels  of  user  expertise  via  prompts  and 
explanations  of  a  suitable  level  of  sophistication 


Chapter  8 

Acceptance  of  IFDIS  by  users 


A  number  of  issues  will  contribute  to  the  acceptance  of  the  system  by 
users.  Quality  of  the  user  interface  will  be  extremely  important  and  should 
be  considered  from  the  stand  point  of  convenience  and  ease  of  use.  Users 
are  much  more  likely  to  accept  the  new  system  if  it  has  obvious  benefits 
over  the  old  way  of  doing  things.  The  role  which  the  new  system  might  play 
in  the  maintenance  environment  will  also  be  important  and  will  determined 
by  the  plan  for  a  Maintenance  Information  Management  System  which  is 
being  developed  by  the  RAAF. 

A  number  of  workers  have  stressed  the  importance  that  an  acceptable 
user  interface  has  in  determining  the  success  of  expert  systems  and  other 
types  of  computer  systems  also.  It  is  extremely  unlikely  that  a  computer 
system  whose  user  interface  is  clumsy  and  inefficient  will  be  accepted  for 
everyday  use  irrespective  of  the  potential  benefits  which  its  use  may  bring. 
The  issues  involved  in  interface  design  are  considered  in  more  detail  in  [17j. 

The  potential  form  which  the  user  interface  could  assume  is  flexible. 
The  availability  of  equipment  to  support  input /output  devices  such  as 
mice,  light-pens,  voice  recognition  and  synthesis  units  and  high  quality 
bit-mapped  graphics  displays  provides  a  wide  choice  of  possibilities.  The 
final  choice  may  be  determined  largely  by  the  environment  in  which  IFDIS 
may  be  required  to  operate.  Situations  in  which  the  level  of  background 
noise  is  high  (e.g.  a  large  hangar)  could  make  the  use  of  voice  recognition 
input  problematical.  The  lack  of  a  convenient  desk-top  at  a  field  location 
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could. make  the  use  of  a  mouse  inconvenient. 

During  a  trip  to  YVilliamtown,  some  of  the  maintenance  personnel  were 
asked  about  their  expectations  of  what  the  user  interface  could  look  like. 
It  seems  that  the  computer  systems  which  they  have  so  far  encountered 
(primarily  the  CAMM  system)  are  oriented  toward  typing  and  text.  Given 
this  type  of  prior  experience  of  computer  systems,  more  modem  mouse/icon 
oriented  interfaces  as  exemplified  by  the  Apple  Macintosh  computer,  should 
be  acceptable.  Most  of  the  more  well-developed  systems  described  in  the 
literature  (e.g  [7])  have  a  good-quality  graphical  interface  on  which  a  repre¬ 
sentation  of  the  deduction  path  being  followed  and  schematics  of  the  system 
etc.  can  be  displayed.  Such  interfaces  require  a  large  development  effort 
but  should  contribute  positively  to  acceptance  and  ease  of  use. 

The  flexibility  of  control  of  the  direction  of  the  dialogue  will  also  affect 
the  user’s  perception  of  the  convenience  of  the  interaction.  The  user  should 
be  able  to  volunteer  as  much  information  as  possible  to  indicate  the  area 
of  investigation  in  which  they  are  interested.  The  EMYCIN  system  is  not 
good  in  this  respect  since  it  is  extremely  difficult,  if  not  impossible  to 
volunteer  information  in  a  realistic  manner  -  you  are  restricted  to  answering 
the  questions  posed  to  you  by  the  system  in  the  order  determined  by  the 
system  and  not  in  the  order  which  may  be  more  convenient  for  you.  An 
indication  of  a  more  flexible  system  for  the  sequencing  of  input  can  be  seen 
in  the  report  by  Slagle  et.  al.  [19j. 

It  is  expected  that  IFDIS  will  be  routinely  consulted  by  maintenance 
personnel  when  a  problem  arises.  The  role  which  IFDIS  will  fill  and  it’s 
relationship  to  the  job  perception  of  the  maintenance  personnel  will  be 
extremely  important.  IFDIS  will  be  used  as  an  advisor/intelligent  assis¬ 
tant  and  maintenance  personnel  will  be  encouraged  to  retain  responsibility 
for  the  diagnosis.  The  benefits  of  using  IFDIS  should  be  such  that  it  be¬ 
comes  just  another  tool  in  the  maintenance  toolbox.  Having  the  power  of  a 
computer  to  help  in  the  location  of  relevant  information  and  to  suggest  al¬ 
ternatives  to  help  the  maintenance  personnel  from  concentrating  too  early 
on  one  specific  possible  cause  of  a  failure  will  be  an  advantage.  The  system 
should  also  be  able  to  suggest  further  tests  which  can  confirm  a  diagnosis  so 
that  the  maintenance  staff  can  be  more  confident  that  the  real  cause  of  the 
problem  has  been  located.  IFDIS  should  be  perceived  to  be  an  intelligent 
assistant  or  advisor  rather  than  a  system  which  will  do  all  the  intelligent 
decision  making  during  the  maintenance  action. 


{27} 


Chapter  9 
Conclusion 


The  potential  benefits  of  developing  a  full  IFDIS  seem  to  be  appreciable. 
A  large  amount  of  further  work  will  be  required  to  fully  realise  the  potential 
of  such  a  system.  It  will  be  necessary  to  involve  personnel  from  the  man¬ 
ufacturer  and  other  services  with  experience  in  maintenance  of  the  F404 
engine  so  that  the  results  of  their  experience  can  be  included  in  the  system. 
Experience  with  the  procedures  specified  in  the  fault-tree  tables  and  pro¬ 
posals  for  shortcuts,  explanations  of  reasoning  etc.  should  be  provided  by 
personnel  with  everyday  experience  with  the  task  at  hand.  Consultations 
with  Australian  personnel,  and  evaluation  by  them  of  a  range  of  possible 
interface  configurations  to  determine  the  appropriate  form  which  the  user 
interface  might  take  will  also  be  essential.  A  great  deal  of  interfacing  work 
to  other  computer  systems  will  also  be  necessary.  Experts  in  the  use  of 
the  CAMM  system  will  need  to  be  consulted  to  determine  the  extent  to 
which  the  CAMM  data  can  be  utilised  and  to  give  indications  of  ways  in 
which  IFDIS  might  be  used  to  control  updates  of  the  CAMM  system  with 
information  from  current  diagnostic  attempts.  Data  on  parts  replaced  etc. 
could  be  gathered  by  IFDIS  in  the  course  of  the  consultation  and  update 
instructions  sent  to  the  CAMM  software  as  part  of  the  diagnostic  consul¬ 
tation.  This  type  of  interaction  between  CAMM  and  IFDIS  will  depend  on 
IFDIS  being  routinely  used  in  all  diagnoses. 

Incorporation  of  models  of  various  types  will  require  further  investiga¬ 
tion.  Some  quantitative  models  of  various  aspects  of  the  engine’s  operation 
are  already  available  but  would  probably  need  to  be  converted  to  run  on  dif- 
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ferent  hardware  to  be  suitable  for  delivery  to  field  units.  The  development 
of  qualitative  models  will  require  further  research. 

The  form  of  the  delivery'  system  will  need  to  be  analysed.  In  certain 
circumstances,  it  is  important  that  the  system  should  be  portable  so  that  it 
can  be  taken  into  the  field.  Use  in  a  field  environment  will  raise  difficulties 
since  problems  with  communication  to  the  UCF  will  result.  Use  of  radio- 
based  data  communications  may  partially  alleviate  this  problem  but  it  will 
probably  be  necessary  to  ensure  that  IFDIS  can  operate  adequately  in  a 
degraded  mode  if  access  to  the  databases  such  as  CAMM  is  not  available. 

The  investigations  undertaken  using  EMYCIN  indicate  that  it  should 
be  feasible  to  construct  a  useful  aid  for  troubleshooting  of  problems  in  the 
F404  engine.  It  will  be  necessary  to  modify  and  supplement  the  knowl¬ 
edge  contained  in  the  fault  tree  tables  to  improve  the  comprehensibility 
of  the  system  to  maintenance  personnel.  Different  inference  strategies  to 
those  provided  by  EMYCIN  will  be  required  to  handle  the  automatically 
recorded  data  and  to  cope  with  mixed-  initiative  dialogue.  The  need  to 
deal  with  uncertain  reasoning  will  probably  also  arise  but  has  not  been 
addressed  in  any  depth  yet.  The  question  of  the  richness  of  the  represen¬ 
tation  environment  also  will  need  further  investigation  when  the  type  of 
knowledge  which  is  to  be  incorporated  into  the  system  becomes  clearer. 
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a.  During  no  start  attempt,  did  fan  and  compressor  rotate? 

b.  Do  substeps  below: 

(1)  In  left  and  right  main  landing  gear  wheeluell, 
observe  fuel  valves  manual  shutoff  arms. 

(2)  Is  engine  fuel  shutoff  valves  manual  shutoff  arms 
in  the  CLOSED  position? 

c.  Do  substeps  below: 

(1)  Open  door  65  L  or  R  (A1-F18AC-LMM-010) 

(2)  Can  compressor  be  rotated  by  turning  power 
transmission  shaft  by  hand? 


c 


o 


b 


v 


d 


I  I 

|  CAUTION  | 


Move  through  engine  inlet  duct  carefully  to 
prevent  damage  to  vortex  generators  on  lower 
surface  midway  down  duct 

d.  Do  substeps  below: 


Fig  1.  Example  of  Fault-Tree  table 
(extracted  from  A1 -F18AC- 270- 200  WP005  00  page  3  ) 


RULE003 


[This  rule  is  tried  in  order  to  find  out  about  whether  the  stall  is 
the  result  of  a  fuel  problem] 

If:  1)  The  main  fuel  control  density  setting  does  not  match  the 

type  of  fuel  in  the  aircraft, 

2)  There  was  indication  of  an  impending  fuel  bypass,  or 

3)  The  compressor  discharge  pressure  line  (ps3)  is  broken 

Then:  It  is  definite  (1.0)  that  the  stall  is  the  result  of  a  fuel 

problem 


Fig.  2.  Example  of  an  IFDIS  rule 


RULE083 


[This  rule  is  tried  in  order  to  find  out  about  the  conclusion] 

If:  1)  You  have  examined  maintenance  status  display  and  recording 

system  maintenance  codes,  and  have  found  one  for  which 
A:  It  is  709,  or 
B:  It  is  759,  and 

2)  A:  The  data  obtained  from  the  signal  data  recording  set 
does  seem  bad, 

B:  Boroscope  inspection  reveals  that  the  hot  section  is  not 
within  limits, 

C:  Signal  data  recording  set  AN/ASM-612  magnetic  tape 
cartridge  engine  data  did  show  engine  INLET  TEMP 
exceeded  38C  when  maintanance  code  709  or  759  was  set, 

D:  The  engine  has  an  fod  problem  as  defined  in  wp7 
tablel , 

E:  More  than  2.0vdc  does  exist  for  the  measurement 
specified  in  question  n  table  1  wp007  0, 

F :  After  starting  both  engines  and  with  both  engines  at 
ground  idle  there  is  more  than  5c  difference  between 
left  and  right  inlet  temp,  or 

G:  There  is  a  problem  with  the  ven  positioning  or  position 
reporting  mechanisms 

Then:  It  is  definite  (1.0)  that  the  following  is  the  conclusion: 

PROBLEM  SOLVED 

Fig.  3  A  rule  with  a  large  number  of  premises 


Appendix  A  -  A  sample  run  of  the  IFDIS  consultant 


All  user  responses  appear  on  lines  which  begin  with 
The  lines  containing  text  enclosed  in  braces  (i.e. 
and  "}")  are  used  as  reference  points  in  the  body  of  the 
report . 

5:33PM  in  <EXT . ARL>EMYCIN . EXE . 3  by  EXT.ARL 

Recording  on  <EXT . ARL>TYPESCRIPT . JENGINE . 4 

Rules,  Parras ,  Go,  etc.  :  Go 
Special  options  (type  ?  for  help) : 


24- Jan-86  17:33:46 

- j ENGINE-2 - 

1)  what  type  of  start  failure  was  it? 

*  *  ? 

What  is  start  failure  type? 

Expected  responses  are:  NO-START,  STALL  or  HOT 
Enter  HELP  for  list  of  user  options. 

**  NO -START 

2)  What  is  serial  number? 

♦♦  1234 

3)  Did  during  the  no  start  attempt,  the  fan  and  compressor  rotate? 

*  *  YES 

4)  Were  the  manual  shutoff  arms  of  the  main  fuel  control  valve  in 
the  closed  position? 

•*  WHY 

[i.e.  WHY  is  it  important  to  determine  whether  the  manual  shutoff 

arms  of  the  main  fuel  control  valve  were  in  the  closed  position?] 

[1.0]  This  will  aid  in  determining  the  conclusion. 

It  has  already  been  established  that 

[1.1]  scart  failure  type  is  no-start 
There  fore ,  i f 

[1.2]  1)  There  is  a  mechanical  problem, 

2)  The  manual  shutoff  arras  of  the  main  fuel  control  valve 

were  in  the  closed  position, 

3)  There  is  a  problem  with  the  igniters,  or 

4)  There  is  a  problem  with  the  fuel  supply  or 

distribution  system 

then 

it  is  definite  (1.0)  that  the  following  is  the  conclusion; 
PROBLEM  SOLVED 

[ALSO : 

display  the  recommended  action] 

[RULE032] 

[back  to  question  4. . 

*  *  -> 

Were  the  manual  shutoff  arms  of  the  main  fuel  control  valve  in  the 
closed  position? 

Expected  responses  are:  YES  or  NO 
Enter  HELP  for  list  of  user  options. 

**  NO 

5)  Does  the  main  igniter  fire  2  to  3  times  per  second  while  the 
engine  is  motoring? 

**  YES 

6)  Does  continuity  exist  with  throttle  at  ground  idle? 

*•  WHY 

[i.e.  WHY  is  it  Important  to  determine  whether  continuity  does  exist 
with  throttle  at  ground  idle?] 

[2.0]  This  will  aid  in  determining  whether  there  is  a  problem  with 
either  the  throttle  or  the  main  fuel  pump. 

If 

[2.1]  1)  Continuity  does  exist  with  throttle  at  ground  idle, 

2)  Continuity  does  not  exist  with  throttle  off,  or 

3)  Fuel  vapors  do  not  come  from  the  exhaust  when  throttle 

is  advanced  during  start,  and 

[2.2]  the  throttle  rig  pin  was  easily  installed  with  the 

throttle  at  mil,  and 


{A} 


(Al> 


[2.3]  the  main  fuel  pump  spline  did  rotate  when  the  power 
transmission  shaft  was  turned 

then 

it  is  definite  (1.0)  that  there  is  a  problem  with  either  the 
throttle  or  the  main  fuel  pump 
[RULE066] 

[back  to  question  6...] 

**  HOW  2.2 

[i.e.  HOW  is  it  determined  whether  the  throttle  rig  pin  was  easily 
installed  with  the  throttle  at  mil?] 

There  are  no  rules  which  allow  the  deduction  of  whether  the 
throttle  rig  pin  was  easily  installed  with  the  throttle  at  mil, 
so  a  question  will  have  to  be  asked. 

[back  to  question  6. . .] 

*  *  ? 

Does  continuity  exist  with  throttle  at  ground  idle? 

Expected  responses  are:  YES  or  NO 
Enter  HELP  for  list  of  user  options. 

*  *  YES 

7)  Was  the  throttle  rig  pin  easily  installed  with  the  throttle  at 
mil? 

**  YES 

8)  Did  the  main  fuel  pump  spline  rotate  when  the  power 
transmission  shaft  was  turned? 

*♦  WHY 

[i.e.  WHY  is  it  important  to  determine  whether  the  main  fuel  pump 

spline  did  rotate  when  the  power  transmission  shaft  was  turned?] 

[3.0]  This  will  aid  in  determining  uhether  there  is  a  problem  with 
either  the  throttle  or  the  main  fuel  pump. 

It  has  already  been  established  that 

[3.1]  continuity  does  exist  with  throttle  at  ground  idle,  and 

[3.2]  the  throttle  rig  pin  was  easily  installed  with  the 

throttle  at  mil 
Therefore,  if 

[3.3]  the  main  fuel  pump  spline  did  rotate  when  the  power 

transmission  shaft  was  turned 

then 

it  is  definite  (1.0)  that  there  is  a  problem  with  either  the 
throttle  or  the  main  fuel  pumD 
[RULE066] 

[back  to  question  8. . .] 

**  HOW  3.2 

[i.e.  HOW  was  it  establish?  *"h..t  the  throttle  rig  pin  was  easily 
installed  with  the  thr. "tie  at  mil?] 

In  answer  to  question  7  you  said  that  the  throttle  rig  pin  was  easily 
installed  with  the  throttle  at  mil. 

Thus,  it  has  been  established  that  the  throttle  rig  pin  was  easily 
installed  with  the  throttle  at  mil. 

[back  to  question  8. . .] 

*  *  ? 

Did  the  main  fuel  pump  spline  rotate-when  the  power  transmission 
shaft  was  turned? 

Expected  responses  are:  YES  or  NO 
Enter  HELP  for  list  of  user  options 
**  YES 

REPLACE  MAIN  FUEL  CONTROL  -  WP008  00. 

9)  Do  give  the  chance  to  go  back  and  change  an  answer  to  explore  a 
different  path  through  the  tree? 

*  * 

**  NO 


Enter  Debug/review  phase,  or  other  option  (?  for  help) 

Rules,  Parms,  Co,  etc.  :  Typescript 

Closing  typescript  file : <EXT.ARL>TYPESCRIPT. JENCINE . 4 


Append i x  3  -  Information  Sources 

Tlie  following  tables  indicate  the  primary  source  of 
information  which  will  be  required  to  answer  a  question. 
Where  data  has  been  gathered  automatically,  it  should  not  be 
necessary  for  this  data  to  be  manually  transcribed  but 
instead  the  data  should  he  collected  directly  by  the  IcDIS 
system.  Questions  which  are  essentially  recommends t ions  for 
specific  maintenance  actions  to  be  performed  are  indicated 
by  ’  as  primary  information  source. 


|  Primary  information 

1 

sources  for  WP0G7  00  Table  1. 

| Quest  ion  Source 

1 

1  a 

1 

MM?  code  from  IECMS 

I  b 

1 

pilot  incident  report  (EE5GC) 

i  c 

1 

" 

!  d 

1 

indication  from  UCE 

!  G 

1 

from  UCF 

!  f 

1 

maintenance  personnel 

i  9 

1 

from  UCE 

h 

1 

maintenance  personnel 

;  L 

1 

UCE 

i  j 

1 

UCF 

!  • 

1 

1 

maintenance  personnel 

i  *n 

1 

1 

CCIJ  or  ground  test 

1 

1 

real-time  hookup 

i  q 

1 

Maintenance  person  r,  e  1  /C  DU 

!  r 

1 

!  s 

1 

| 

real-time  hookup 

i  u 

1 

.. 

i  v 

1 

maintenance  personnel 

i  y 

1 

1 

1 

CDU  or  ground  test 

M 

•  1 

i 

|  a  n 

■  a  n 

1 

1 

[ 

II 

>1 

i  ic 

1 

- 

j  ,vi 

1 

CDU  or  ground  tost 

|  ao 

1 

- 

i  a  f 

1 

CDU  or  ground  test 

|  an 

1 

i  ah 

1 

- 

!  a  i 

1 

CDU  or  ground  test 

i  a  j 

|  uk 

I  a  i 

|  am 

! 

1 

CDU  or  ground  test 

1 

1 

maintenance  personnel 

|  an 

1 

M  II 

|  ao 

1 

•  |  11 

i  up 

1 

- 

l  <*q 

1 

" 

1  ar 

1 

maintenance  personnel 

|  as , at . an , av , aw , ax , ay 

1. 

1 

X 


(Primary  information  sources  for  WPOOV  OO  Table  z' 


Quest  ion  | 


Source 


- 1 


a 

b 


;u) 

| 

.Ki  | 

'HI 


«'i‘  J 
■  ih 
.1  1 


|  r oa  1  -  r.  i  me  hookup 

J  maintenance  personnel 
\  CDU  or  ground  tost 

J  CDi  J  or  ground  tost 

i 

|  maintenance  personnel 
!  bell  or  p  ound  test 

|  na  intensr-.ee  personnel 


!  CD  'J  or  ground  test 


i 


i 


|  CDU  or  yround  test 


urn  | 


| Primary 

1 

information  sources  for  V/P007  OO  Table  3. 

| Question 

|  Source 

1  el 

|  CDU  or  ground  test 

1  b 

|  rc-al -time  hookup 

1  C 

|  maintenance  sersor.nel 

1  d 

1 

1  e 

|  CDU  or  ground  test 

1  f 

1  " 

I  g 

J  maintenance  personnel 

1  h 

|  real-t^m.e  hookup 

l  i 

|  maintenance  personnel 

1  j 

i  ' 

1 

1  ' 

1  l 

1  ' 

1  m 

|  CDU  or  ground  tost 

1  n 

1  " 

l  o 

1  " 

i  P 

I  " 

!  r 

1  - 

i  s 

I  - 

i  ^ 

1  - 
!  - 

1  ,  r 

I  ' 

1 

I  - 

i  CDU  or  ground  test 

!  X 

|  CDU  or  ground  test 

i  y 

i  - 

1  - 

[  CDU  ur  ground  test 

j  a  a 

1  - 

|  ab 

|  CDU  or  around  test 

|  dC 

1  - 

|  ad 

1 

i  - 

1 
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